Range differentiators for auto-focusing in optical imaging systems

ABSTRACT

A range differentiator useful for auto-focusing, the range differentiator including an image generator providing an image of a scene at various physical depths, a depth differentiator distinguishing portions of the image at depths below a predetermined threshold, irrespective of a shape of the portions, and providing a depth differentiated image and a focus distance ascertainer ascertaining a focus distance based on the depth differentiated image.

REFERENCE TO RELATED APPLICATIONS

Reference is hereby made to U.S. Provisional Patent Application No.62/634,870, entitled RANGE DIFFERENTIATORS FOR AUTO-FOCUSING IN OPTICALIMAGING SYSTEMS, filed Feb. 25, 2018, the disclosure of which is herebyincorporated by reference and priority of which is hereby claimed,pursuant to 37 CFR 1.78(a)(4) and 5(i).

FIELD OF THE INVENTION

The present invention relates generally to optical imaging systems andmore particularly to systems and methods useful for auto-focusing inoptical imaging systems.

BACKGROUND OF THE INVENTION

Various types of auto-focusing systems for use in optical imagingsystems are known in the art.

SUMMARY OF THE INVENTION

The present invention seeks to provide systems and methods relating todepth range differentiation for use in auto-focusing in optical imagingsystems.

There is thus provided in accordance with a preferred embodiment of thepresent invention a range differentiator useful for auto-focusing, therange differentiator including an image generator providing an image ofa scene at various physical depths, a depth differentiatordistinguishing portions of the image at depths below a predeterminedthreshold, irrespective of a shape of the portions, and providing adepth differentiated image and a focus distance ascertainer ascertaininga focus distance based on the depth differentiated image.

In accordance with a preferred embodiment of the present invention theimage generator includes a feature specific illuminator for illuminatingthe scene during acquisition of the image. Additionally, the depthdifferentiator is operative to distinguish between the portions of theimage at depths below the predetermined threshold and portions of theimage at depths at or above the predetermined threshold based ondifferences in optical properties therebetween, under illumination bythe feature specific illuminator.

In accordance with a preferred embodiment of the present invention thefeature specific illuminator includes a UV illumination source and thedepth differentiator is operative to distinguish between the portions ofthe image based on differences in fluorescence therebetween.Alternatively, the feature specific illuminator includes dark field andbright field illumination sources and the depth differentiator isoperative to distinguish between the portions of the image based ondifferences in reflectance therebetween.

Preferably, the focus distance ascertainer is operative to ascertain thefocal distance based on one of the portions of the image at depths belowthe predetermined threshold and the portions of the image at a depth ator above the predetermined threshold.

In accordance with a preferred embodiment of the present invention therange differentiator also includes an image focus analyzer operative toprovide a focus score based on portions of the image at a depth at orabove the predetermined threshold and the focus distance ascertainer isoperative to ascertain the focus distance based on the focus score.Additionally, the image focus analyzer includes an illuminator forilluminating the scene with illumination for enhancing an imaged textureof the portions of the image at a depth at or above the predeterminedthreshold. Additionally, the illuminator includes a dark fieldilluminator. Alternatively or additionally, the focus score is assignedirrespective of a shape of the portions. In accordance with a preferredembodiment of the present invention the focus score is individuallyassigned for each pixel corresponding to the portions of the image at adepth at or above the predetermined threshold.

Preferably, the portions of the image at a depth at or above thepredetermined threshold are machine identifiable.

In accordance with a preferred embodiment of the present invention theimage generator includes a camera and the depth differentiated imageincludes a two-dimensional image of the scene. Additionally oralternatively, the image generator includes a plenoptic camera and thedepth differentiated image includes a three-dimensional image of thescene. In accordance with a preferred embodiment of the presentinvention the feature specific illuminator includes a dark fieldilluminator.

In accordance with a preferred embodiment of the present invention theimage generator includes a projector projecting a repeating pattern ontothe scene and the depth differentiator includes a phase analyzeroperative to analyze shifts in phase of the repeating pattern and derivea map of the physical depths based on the shifts in phase, the mapforming the depth differentiated image. Additionally, the focus distanceascertainer is operative to ascertain the focus distance based on atleast one of the physical depths.

In accordance with a preferred embodiment of the present invention therepeating pattern includes at least one of a sinusoidal repeatingpattern and a binary repeating pattern. Additionally, the repeatingpattern has a sufficiently low spatial frequency such that the phaseanalyzer is operative to uniquely correlate the shifts in phase to thephysical depths. Additionally or alternatively, the map of the physicaldepths is one of a two dimensional map and a three dimensional map.

There is also provided in accordance with another preferred embodimentof the present invention a range differentiator useful forauto-focusing, the range differentiator including an image generatorproviding an image of a scene at various physical depths, a depthdifferentiator distinguishing portions of the image at depths below apredetermined threshold, an image focus analyzer operative to provide afocus score based on portions of the image at a depth at or above thepredetermined threshold and a focus distance ascertainer ascertaining afocus distance based on the focus score.

In accordance with a preferred embodiment of the present invention theimage generator includes a feature specific illuminator for illuminatingthe scene during acquisition of the image. Additionally, the featurespecific illuminator includes a UV illumination source and the depthdifferentiator distinguishes portions of the image based on differencesin fluorescence therebetween. Alternatively, the feature specificilluminator includes a combined dark field and bright field illuminatorand the depth differentiator distinguishes portions of the image basedon differences in reflectance therebetween.

In accordance with a preferred embodiment of the present invention theimage focus analyzer includes an illuminator for illuminating the scenewith illumination for enhancing an imaged texture of the portions of theimage at a depth at or above the predetermined threshold. Additionally,the illuminator includes a dark field illuminator. Additionally oralternatively, the illuminator and the feature specific illuminatorshare at least one common illumination component.

In accordance with a preferred embodiment of the present invention thefocus score is assigned irrespective of a shape of the portions.Additionally or alternatively, the focus score is individually assignedfor each pixel corresponding to the portions of the image at a depth ator above the predetermined threshold.

Preferably, the portions of the image at a depth at or above thepredetermined threshold are machine identifiable.

There is further provided in accordance with yet another preferredembodiment of the present invention a range differentiator useful forauto-focusing, the range differentiator including a target identifierincluding a user interface enabling a user to identify a machineidentifiable feature of an object in an image, a feature detectoroperative to identify at least one occurrence of the machineidentifiable feature in an image irrespective of a shape of the featureand a focus distance ascertainer ascertaining a focal distance to themachine identifiable feature.

Preferably, the range differentiator also includes a feature specificilluminator for illuminating the object during acquisition of the image.

In accordance with a preferred embodiment of the present invention thefeature specific illuminator includes a UV illumination source and thefeature identifier identifies the machine identifiable feature based onfluorescence thereof. Alternatively, the feature specific illuminatorincludes a combined dark field and bright field illuminator and thefeature identifier identifies the machine identifiable feature based onreflectance thereof.

In accordance with a preferred embodiment of the present invention rangeascertainer includes an illuminator for illuminating the object withillumination for enhancing an imaged texture of the feature of theobject in the image. Additionally, the illuminator includes a dark fieldilluminator.

Preferably, the illuminator and the feature specific illuminator shareat least one common illumination component.

In accordance with a preferred embodiment of the present invention thefeature of the object includes a conductive feature. Additionally, thefeature of the object includes an indent in the conductive feature.

There is yet further provided in accordance with still another preferredembodiment of the present invention a range differentiator useful forauto-focusing, the range differentiator including a first imagegenerator including a first imaging modality and providing a first imageof a scene at various physical depths, a depth differentiatordistinguishing portions of the first image at depths below apredetermined threshold and providing a depth differentiated image, afocus distance ascertainer ascertaining a focal distance based on thedepth differentiated image and a second image generator including asecond imaging modality and providing a second image of the sceneautomatically focused at the focal distance.

In accordance with a preferred embodiment of the present invention thefirst imaging modality includes combined bright and dark fieldillumination and the second imaging modality includes dark fieldillumination. Additionally, the second image generator includes aplenoptic camera.

In accordance with a preferred embodiment of the present invention thefirst imaging modality includes dark field illumination and the secondimaging modality includes combined bright and dark field illumination.Additionally, the first image generator includes a plenoptic camera.

There is still further provided in accordance with still anotherpreferred embodiment of the present invention a range differentiatoruseful for auto-focusing, the range differentiator including a projectorprojecting a repeating pattern onto an object including features ofvarious physical depths, a sensor acquiring an image of the objecthaving the repeating pattern projected thereon, a phase analyzeranalyzing shifts in phase of the repeating pattern and deriving a map ofthe physical depths of the features based on the shifts in phase and afocus analyzer ascertaining a focus distance to at least one of thefeatures.

In accordance with a preferred embodiment of the present invention therepeating pattern includes at least one of a sinusoidal repeatingpattern and a binary repeating pattern. Additionally or alternatively,the repeating pattern has a sufficiently low spatial frequency such thatthe phase analyzer is operative to uniquely correlate the shifts inphase to the physical depths.

Preferably, the map of the physical depths is one of a two dimensionalmap or a three dimensional map.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood and appreciated more fully fromthe following detailed description, taken in conjunction with thedrawings in which:

FIG. 1 is a simplified schematic illustration of an optical imagingsystem including auto-focusing functionality, constructed and operativein accordance with a preferred embodiment of the present invention;

FIG. 2 is a simplified block-diagram representation of elements of asystem of the type illustrated in FIG. 1;

FIGS. 3A, 3B and 3C are simplified images produced by a system of thetype illustrated in FIGS. 1 and 2, respectively showing an initial imageof an object acquired under feature-specific illumination conditions, asegmented image of the object based on the initial image and useful forautofocusing and an auto-focused image of the object based on thesegmented image;

FIG. 4 is a simplified graph illustrating trends in variables useful forgenerating the auto-focused image of the type shown in FIG. 3C;

FIG. 5 is a simplified schematic illustration of an optical imagingsystem including auto-focusing functionality, constructed and operativein accordance with another preferred embodiment of the presentinvention;

FIG. 6 is a simplified block-diagram representation of elements of asystem of the type illustrated in FIG. 5;

FIGS. 7A, 7B and 7C are simplified images produced by a system of thetype illustrated in FIGS. 5 and 6, respectively showing an initial imageof an object acquired under feature-specific illumination conditions, asegmented image of the object based on the initial image and useful forautofocusing and an auto-focused image of the object based on thesegmented image;

FIGS. 8A, 8B and 8C are simplified images produced by a system of any ofFIGS. 1-2 and 5-6, respectively illustrating an initial image of anadditional feature acquired under feature-specific illuminationconditions, a depth-differentiated image based on the initial image anduseful for autofocusing and an auto-focused image based on thedepth-differentiated image;

FIG. 9 is a simplified graph illustrating trends in variables useful forgenerating the auto-focused image of the type shown in FIG. 8C;

FIG. 10 is a simplified schematic illustration of an optical processingsystem including auto-focusing functionality, constructed and operativein accordance with a further preferred embodiment of the presentinvention;

FIGS. 11A, 11B, 11C and 11D are simplified images produced by a systemof the type illustrated in FIG. 10, respectively showing an initialimage of an object acquired under feature-specific illuminationconditions, a depth-differentiated image based on the initial image anduseful for autofocusing, a two-dimensional auto-focused image based onthe depth-differentiated image and a three-dimensional image;

FIGS. 12A, 12B, 12C and 12D are simplified images additionally oralternatively produced by a system of the type illustrated in FIG. 10,respectively showing an initial three-dimensional image of an objectacquired under feature-focusing illumination conditions, a correspondingtwo-dimensional image, a depth-differentiated three-dimensional imagebased on the initial image and a two-dimensional auto-focused imagebased on the depth-differentiated three-dimensional image;

FIG. 13 is a simplified schematic illustration of an optical processingsystem including auto-focusing functionality, constructed and operativein accordance with a yet a further preferred embodiment of the presentinvention; and

FIGS. 14A, 14B and 14C are simplified images produced by a system of thetype illustrated in FIG. 13, respectively showing an initial image of anobject acquired under first illumination conditions and two-dimensionaland three-dimensional height-mapped images based on the initial imageand useful for auto-focusing.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Reference is now made to FIG. 1, which is a simplified illustration ofan optical imaging system including auto-focusing functionality,constructed and operative in accordance with a preferred embodiment ofthe present invention, and to FIG. 2, which is a simplifiedblock-diagram representation of elements of a system of the typeillustrated in FIG. 1.

As seen in FIGS. 1 and 2, there is provided an optical imaging system100, preferably including an optical imaging head 102 mounted on achassis 104. Chassis 104 preferably includes a table 106 adapted forplacement thereon of an object 108 to be imaged. Optical imaging system100 is preferably operative to provide an image of object 108, forexample for the purposes of inspection or processing of object 108.

Object 108 is preferably a non-planar object comprising physicalfeatures at more than one physical depth. Here, by way of example,object 108 is shown to be embodied as a PCB including a non-conductivesubstrate 109 having metallic traces 110 formed thereon, which metallictraces 110 may be embedded or may protrude with respect to a surface ofsubstrate 109. It is appreciated, however, that optical imaging head 102may be used to acquire images of any suitable target or scene havingphysical features at more than one physical height or depth including,but not limited to, PCBs, wafer dies, assembled PCBs, flat paneldisplays and solar energy wafers.

In some cases, it may be desirable to generate a focused image of afeature of interest included in object 108, which feature of interest isat a different physical height or depth with respect to other featuresof object 108. For example, in the case of object 108, it may bedesirable to generate an image in which metallic traces 110 are in focusfor the purposes of inspection thereof. It is a particular feature of apreferred embodiment of the present invention that optical imagingsystem 100 includes a range differentiator 120 providing depthdifferentiated images and thereby enabling auto-focusing on a feature ofinterest, such as metallic traces 110, notwithstanding the difference inphysical depth between the feature of interest and other features, suchas substrate 109. Furthermore, such auto-focusing may be achieved byrange differentiator 120 irrespective of a shape of the feature ofinterest.

As seen most clearly in FIG. 2, range differentiator 120 preferablyincludes an image generator operative to provide an image of a scene atvarious physical depths, here embodied, by way of example, as includingillumination module 122 for illuminating object 108. Illuminationprovided by illumination module 122 is preferably directed towardsobject 108 by way of a movable lens portion 124, which moveable lensportion 124 is preferably mounted on a translation stage 126 controlledby a controller 128. Light emanating from object 108 is preferablydirected by way of moveable lens portion 124 towards a camera sensor130, which camera sensor 130 is preferably coupled to a processor 132.

Range differentiator 120 preferably operates in two modes. In a firstmode of operation of range differentiator 120, object 108 is preferablyimaged by camera sensor 130 under illumination conditions in which thefeature of interest is clearly distinguishable from the other featuresof object 108 having a different physical depth than the feature ofinterest. Such imaging is preferably carried out following an initialcoarse focusing of camera sensor 130 on object 108, such that the imageacquired thereby is in sufficiently good focus for subsequentprocessing.

Illumination under which the feature of interest is clearlydistinguishable from the other features of object 108 having a differentphysical depth than the feature of interest may be termed featurespecific illumination and may be provided by a feature specificilluminator 140 included in illumination module 122. Here, by way ofexample only, feature specific illuminator 140 is shown to be embodiedas a UV light source, preferably providing very short wavelengthillumination having a wavelength of less than or equal to approximately420 nm.

Under UV illumination provided by feature specific illuminator 140,non-conductive substrate 109 fluoresces whereas metallic traces 110 donot. An exemplary image of substrate 109 and metallic traces 110 thereonunder UV feature specific illumination conditions is shown in FIG. 3A.As seen in FIG. 3A, non-conductive substrate 109 has a bright appearancedue to the fluorescence thereof whereas metallic traces 110 have a darkappearance. Non-conductive substrate 109 is thus clearly distinguishablefrom metallic traces 110 in the image of FIG. 3A. Furthermore, as aresult of the fluorescence of substrate 109, additional features ofobject 108 that may lie beneath the surface of substrate 109 are maskedand thereby do not appear in the image of FIG. 3A, thus simplifyingsubsequent image processing.

Following the generation of an initial feature specific image, such asthat shown in FIG. 3A, a tagged or segmented image is preferablygenerated, which segmented image is based on the initial featurespecific image. An exemplary segmented image based on the featurespecific image of FIG. 3A is illustrated in FIG. 3B. In the segmentedimage of FIG. 3B, pixels corresponding to dark metallic traces 110 aremarked in gray, identifying these pixels as corresponding to regions ofinterest, and pixels corresponding to bright substrate regions 109 aremarked in white, identifying these pixels as corresponding to regions ofnon-interest, which regions of non-interest are to be ignored insubsequent image processing steps. Pixels corresponding to regions ofunclear identity are marked in black, such as in a region 112,identifying these pixels as corresponding to regions of questionableinterest, which regions are also to be ignored in subsequent imageprocessing steps. Preferably, a predetermined threshold for level ofpixel brightness in FIG. 3A may be applied in order to distinguishbetween dark pixels corresponding to metallic traces 110 and brightpixels corresponding to background substrate 109.

It is understood that the segmented image of FIG. 3B thus effectivelyforms a depth differentiated mask image, in which portions of thefeature specific image of FIG. 3A at or below a given depth, here, byway of example comprising substrate 109, are distinguished from portionsof the feature specific image of FIG. 3A above the given depth, here, byway of example, comprising metallic traces 110. It is appreciated thatthe differentiation between portions of the feature specific image ofFIG. 3A at different physical depths is based on the difference inoptical properties therebetween, and more specifically the difference influorescence under UV illumination therebetween, and is independent andirrespective of the physical shapes of the features.

The generation of the segmented mask image of FIG. 3B may beautomatically carried out by computing functionality included in system100, here embodied, by way of example only, as processor 132, whichprocessor 132 may be included in a computer 144. It is appreciated thatprocessor 132 thus preferably operates as a depth differentiator,operative to distinguish portions of an initial feature specific image,such as the image of FIG. 3A, at depths below a predetermined threshold,irrespective of a shape of the portions, and to provide a depthdifferentiated image, such as the depth differentiated image of FIG. 3B.

It is further appreciated that feature specific UV illuminator 140 incombination with sensor 130 and processor 132 constitute a particularlypreferred embodiment of an image generator, providing an image of object108 including substrate 109 and metallic traces 110. It is understood,however, that the image generation functionality of range differentiator120 is not limited to the particular camera and illumination componentsdescribed herein and rather may comprise any suitable componentsfunctional to generate an image of a scene at various physical depths,in which features having different physical depths are differentiablebased on the optical properties thereof and irrespective of the shapethereof.

Computer 144 may include a user interface, enabling a user to identifythe feature of interest in the feature specific image, such as metallictraces 110 in FIG. 3A. It is appreciated that the feature of interestmay be identifiable by a user as well as preferably being a machineidentifiable feature, the presence of which machine identifiable featurein the feature specific image may be detected based on the appearancethereof, irrespective of the shape of the feature, by computer 144. Itis appreciated that computer 144 therefore may operate both as a targetidentifier, enabling a user to identify a machine identifiable feature,and as a feature detector, for preferably automatically identifying themachine identifiable feature.

In a second mode of operation of range differentiator 120, following thegeneration of a segmented image, such as that shown in FIG. 3B, object108 is preferably imaged by camera sensor 130 under illuminationconditions best suited for enhancing the imaged texture of the featureof interest, which feature of interest is here embodied as metallictraces 110. Such illumination may be termed feature focusingillumination and is preferably provided by a feature focusingilluminator 150 included in illumination module 122. Here, by way ofexample only, feature focusing illuminator 150 is shown to be embodiedas a bright field illuminator.

It is appreciated that although feature specific illuminator 140 andfeature focusing illuminator 150 are shown herein to be embodied as twoseparate illuminators included in illumination module 122, featurespecific illuminator 140 and feature focusing illuminator 150 mayalternatively be provided by at least partially common illuminationelements having at least partially overlapping functionality, forproviding both feature specific and feature focusing illumination, as isexemplified hereinbelow with reference to FIG. 6.

During the imaging of object 108 under lighting provided by featurefocusing illuminator 150, the vertical position of lens 124 with respectto object 108 is preferably incrementally shifted, such that a focalheight of lens 124 with respect to object 108 is correspondinglyadjusted. Adjustment of lens 124 may be controlled by controller 128,which controller 128 is preferably operative to incrementally move stage126, and thereby lens 124, with respect to object 108. Additionally oralternatively, the focal height of lens 124 with respect to object 108may be adjusted by way of adjustment to the height of table 106 and/orof optical head 102 in its entirety.

For each position of lens 124, an image of object 108 is preferablyacquired by sensor 130. A series of images at a range of focal heightsof lens 124 above object 108 is thus preferably generated. An imagefocus analyzer, preferably embodied as processor 132, is preferablyoperative to perform image focus analysis on the series of images, inorder to provide a focus score based on portions of each image, at adepth at or above a predetermined depth and to ascertain a focusdistance based on the focus score. It is appreciated that processor 132thus additionally preferably operates as a focus distance ascertainer,ascertaining a focus distance based on a depth differentiated image,such as the image of FIG. 3B.

The focus score is preferably calculated for each image acquired underlighting conditions provided by feature focusing illuminator 150, thefocus score being based only on those pixels identified in the segmenteddepth differentiated image, such as the image of FIG. 3B, ascorresponding to regions of interest. In the case of metallic traces 110on substrate 109, by way of example, each pixel identified in the depthdifferentiated image, such as the image of FIG. 3B, as corresponding toregions of interest, such as metallic traces 110, is assigned a focusmeasure based on the local texture. Such a focus measure may be, by wayof example, the gradient magnitude at the pixel neighborhood, or may beany other focus measure known in the art.

Pixels identified in the depth differentiated image, such as the imageof FIG. 3B, as corresponding to regions of non-interest, such assubstrate 109, are preferably assigned a focus measure of zero. Theoverall focus score of each image acquired under lighting conditionsprovided by feature focusing illuminator 150 is preferably given by thesum of the focus measures of all of the individual pixels in the imagecorresponding to the region of interest, such as the metallic traces110. Since the focus measures of pixels corresponding to regions ofnon-interest, such as substrate 109, is set to zero, pixelscorresponding to regions of non-interest do not contribute to theoverall focus score of the image and are effectively ignored in thefocus score calculation.

It is appreciated that in the above described embodiment the focus scorefor each image is thus preferably based only on those portions of theimage at a depth equal to or above the predetermined depth, in this casecorresponding to the depth of metallic traces 110, and does not takeinto account those portions of the image below the predetermined depth,in this case corresponding to substrate 109. Alternatively, the focusscore may be calculated based only on those portions of the depthdifferentiated image at a depth below a predetermined depth, for examplein the case of a feature of interest being embedded within a substrate.

The focus score obtained for each image may be plotted as a function ofthe focal height of lens 124, as illustrated in FIG. 4. The lensposition at which the feature of interest is in optimum focus may beidentified as that lens position corresponding to the image having thehighest focal score. In the case of data presented in FIG. 4, thehighest focal score of 80 is seen to correspond to a focal height ofapproximately 6487 μm. A representative image having the highest focusscore, in which metallic traces 110 are in best focus, is seen in FIG.3C. As appreciated from consideration of the focused image of FIG. 3C,the texture of metallic traces 110 is highly visible whereas substrate109 appears smooth, since the image of FIG. 3C has been acquired at afocal height optimum for focus on metallic traces 110 without takinginto account substrate 109, which substrate 109 is at a differentphysical height than metallic traces 110.

It is appreciated that the optimum focal height corresponding to thefocal height of the image having the highest focus score, is preferablyfound to an accuracy greater than the height step between consecutiveimages. This may be achieved by any method suitable for finding themaximum of a function, such as, by way of example only, fitting the datain the region close to the maximum to a parabolic function.

It is further appreciated that the feature specific illumination,preferably provided by feature specific illuminator 140, is not limitedto UV illumination and may be any type of illumination under whichtarget features of different physical depths exhibit a correspondinglydifferent optical response and hence may be distinguished between in animage thereof. By way of example, UV feature specific illuminator 140may be replaced by an alternative illuminator, as seen in the embodimentof FIGS. 5 and 6.

Turning now to FIGS. 5 and 6, an optical imaging system 500 may beprovided generally resembling optical imaging system 100 in relevantaspects thereof, with the exception of UV feature specific illuminator140 of illuminator 122 of range differentiator 120 being replaced by acombined bright and dark field illuminator or broad angle illuminator540, as seen in FIG. 6. Feature specific illuminator 540 may be of atype generally described in Chinese patent application 201510423283.5,filed Jul. 17, 2015, or other illuminators known in the art.

Here, by way of example only, object 108 is shown to be embodied as aPCB 508 including a laminate region 509 having copper traces 510 formedthereon and protruding with respect thereto. For example, in the case ofPCB 508, it may be desirable to generate an image in which copper traces510 are in focus for the purposes of inspection thereof.

Under combined bright and dark field illumination or broad angleillumination provided by feature specific illuminator 540, laminateregion 509 is significantly less reflective than copper traces 510. Anexemplary image of laminate region 509 and copper traces 510 underfeature specific reflective illumination conditions provided by featurespecific illuminator 540 is shown in FIG. 7A. As seen in FIG. 7A,laminate region 509 has a dark appearance due to the lower reflectivitythereof whereas copper traces 510 have a bright appearance. Laminateregion 509 is thus clearly distinguishable from copper traces 510 in theimage of FIG. 7A. Furthermore, as a result of the opaque appearance oflaminate 509, additional features of object 508 that may lie beneathlaminate 509 are masked and thereby do not appear in the image of FIG.7A, thus simplifying subsequent image processing.

A depth differentiated or segmented image based on the initial featurespecific image of FIG. 7A is shown in FIG. 7B. In the segmented image ofFIG. 7B, pixels corresponding to bright copper traces 510 are marked inwhite, identifying these pixels as corresponding to regions of interest,and pixels corresponding to dark laminate region 509 are marked inblack, identifying these pixels as corresponding to regions ofnon-interest, which regions of non-interest are to be ignored insubsequent image processing steps. Preferably, a predetermined thresholdfor level of pixel brightness may be applied in order to distinguishbetween white pixels corresponding to copper traces 510 and black pixelscorresponding to laminate 509.

It is understood that the segmented image of FIG. 7B thus effectivelyforms a depth differentiated image, in which portions of the featurespecific image of FIG. 7A at depths below a given threshold, here, byway of example comprising laminate 509, are distinguished from portionsof the feature specific image of FIG. 7A at depths at or above the giventhreshold, here, by way of example, comprising copper traces 510. It isappreciated that the differentiation between portions of the featurespecific image of FIG. 7A at different physical depths is based on thedifference in optical properties therebetween, and more specifically thedifference in reflectance under combined bright and dark field or broadangle illumination therebetween, and is independent of the physicalshapes of the features.

The generation of the segmented mask image of FIG. 7B may beautomatically carried out by computing functionality included in system500, here embodied, by way of example only, as processor 132, whichprocessor 132 may be included in computer 144. It is appreciated thatprocessor 132 thus preferably operates as a depth differentiator withinsystem 500, operative to distinguish portions of an initial featurespecific image, such as the image of FIG. 7A, at depths below apredetermined threshold, irrespective of a shape of the portions, and toprovide a depth differentiated image based thereon, such as the depthdifferentiated image of FIG. 7B.

The acquisition of a series of images under illumination conditionsprovided by feature focusing illumination 150 and the subsequentpreferably automated selection of an image in which copper traces 510are best in focus at an optimal focus distance, based on a comparison offocal scores assigned only to pixels corresponding to copper traces 510identified in the segmented, depth differentiated image, such as theimage of FIG. 7B, is generally as described above with reference toFIGS. 3B-4. Generally in the manner as described hereinabove withreference to FIG. 4, processor 132 within system 500 preferablyadditionally operates as a focus distance ascertainer, ascertaining afocus distance based on a depth differentiated image, such as the imageof FIG. 7B.

An image of object 508 assigned the highest focus score, in whichmetallic traces 510 are thus in optimum focus, is seen in FIG. 7C. It isappreciated that the focus score is here preferably calculated basedonly on those portions of the depth differentiated image, such as theimage of FIG. 7B, at a depth at or above a predetermined depththreshold, here corresponding to protruding copper traces 510.Alternatively, the focus score may be calculated based only on thoseportions of the depth differentiated image at a depth below apredetermined depth threshold, for example in the case of a feature ofinterest being embedded within a substrate.

It is appreciated that the automatically focused images generated by thesystems of FIGS. 1-2 and 5-6, such as images shown in FIGS. 3C and 7C,correspond to images obtained at a focal distance such that a particularfeature of interest of the object being imaged is in best focus,notwithstanding the difference in physical height or depth between theparticular feature of interest and other features that may form a partof the object being imaged.

However, systems of the present invention may alternatively be operativeto automatically generate a range image of an object or scene, in orderto obtain a depth profile of a particular feature of interest of theobject or scene to be imaged, which feature of interest preferably has aphysical depth or height differing from the depth or height of otherfeatures forming a part of the object or scene to be imaged.

The operation of a system of the type shown in FIGS. 5 and 6 is nowdescribed in relation to the generation of a range image of an object1108. Object 1108 may include a non-conductive substrate 1109 having acopper region 1110 formed thereon, images of which object 1108 are shownin FIGS. 8A-8C. The system of FIGS. 5 and 6 is preferably operative toautomatically generate a range image of copper region 1110, which copperregion 1110 may protrude or be recessed with respect to substrate 1109.Such a range image may be useful, for example, in detecting the presenceand measuring the depth of indents within copper region 1110. It isappreciated that although the generation of a range image is describedhereinbelow with reference to the system of FIGS. 5 and 6, any of thesystems described hereinabove may alternatively be configured to providea range image of a feature of interest, with appropriate modificationsas will be evident to one skilled in the art.

In the first mode of operation of range differentiator 120 in system500, object 1108 is preferably imaged by camera sensor 130 underillumination conditions in which the feature of interest is clearlydistinguishable from the other features of object 1108 having adifferent physical depth than the feature of interest. An exemplaryimage of substrate 1109 and copper region 1110 thereon under featurespecific illumination conditions is shown in FIG. 8A. As seen in FIG.8A, non-conductive substrate 1109 has a dark appearance due to the lowreflectance thereof whereas copper region 1110 has a brighterappearance. Non-conductive substrate 1109 is thus clearlydistinguishable from copper region 1110 in the image of FIG. 8A.Furthermore, as a result of the opaque appearance of substrate 1109,additional features of object 1108 that may lie beneath substrate 1109are masked and thereby do not appear in the image of FIG. 8A, thussimplifying subsequent image processing.

Following the generation of an initial feature specific image, such asthat shown in FIG. 8A, a depth differentiated or segmented image ispreferably generated, which segmented image is based on the initialfeature specific image. An exemplary segmented image based on thefeature specific image of FIG. 8A is illustrated in FIG. 8B. In thesegmented image of FIG. 8B, pixels corresponding to bright copper region1110 are marked in white, identifying these pixels as corresponding to aregion of interest, and pixels corresponding to dark substrate regions1109 are marked in black, identifying these pixels as corresponding toregions of non-interest, which regions of non-interest are to be ignoredin subsequent image processing steps. Preferably, a predeterminedthreshold for level of pixel brightness may be applied in order todistinguish between bright pixels corresponding to copper region 1110and dark pixels corresponding to background substrate 1109.

It is understood that the segmented image of FIG. 8B thus effectivelyforms a depth differentiated image, in which portions of the featurespecific image of FIG. 8A at depths below a given threshold, here, byway of example comprising substrate 1109, are distinguishable fromportions of the feature specific image of FIG. 8A at depths at or abovea given threshold, here, by way of example, comprising copper region1110.

It is appreciated that the differentiation between portions of thefeature specific image of FIG. 8A at different physical depths is basedon the difference in optical properties therebetween, and morespecifically the difference in reflectance under appropriateillumination therebetween, and is independent of the physical shapes ofthe features.

The generation of the segmented mask image of FIG. 8B may beautomatically carried out by processor 132, which processor 132 may beincluded in computer 144. It is appreciated that processor 132 thuspreferably operates as a depth differentiator, operative to distinguishportions of an initial feature specific image, such as the image of FIG.8A, at depths below a predetermined threshold, irrespective of a shapeof the portions, and to provide a depth differentiated image, such asthe depth differentiated image seen in FIG. 8B, based thereon.

It is further appreciated that feature specific illuminator 540 incombination with sensor 130 and processor 132 thus constitutes apreferred embodiment of an image generator, providing an image of object1108 including substrate 1109 and copper region 1110.

Computer 144 may include a user interface, enabling a user to identifythe feature of interest in the feature specific image, such as copperregion 1110 in FIG. 8A. It is appreciated that the feature of interestmay be identifiable by a user as well as preferably being a machineidentifiable feature, the presence of which machine identifiable featurein the feature specific image may be detected based on the appearancethereof, irrespective of the shape of the feature. It is appreciatedthat computer 144 therefore may operate both as a target identifier,enabling a user to identify a machine identifiable feature, and as afeature detector, for preferably automatically identifying the machineidentifiable feature.

In the second mode of operation of range differentiator 120, followingthe generation of a segmented depth differentiated image, such as thatshown in FIG. 8B, object 1108 is preferably imaged by camera sensor 130under illumination conditions best suited for generating a depth profileof the feature of interest, which feature of interest is here embodiedas copper region 1110.

During the imaging of object 1108 under lighting provided by featurefocusing illuminator 150, the vertical position of lens 124 with respectto object 1108 is preferably incrementally shifted, such that focalheight of lens 124 with respect to object 1108 is correspondinglyadjusted. Adjustment of lens 124 may be controlled by controller 128,which controller 128 is preferably operative to incrementally move stage126, and thereby lens 124, with respect to object 1108. Additionally oralternatively, the focal height of lens 124 with respect to object 1108may be adjusted by way of adjustment to the height of table 106 and/orof optical head 102 in its entirety.

For each position of lens 124, an image of object 1108 is preferablyacquired by sensor 130. A series of images at a range of focal heightsof lens 124 above object 108 is thus preferably generated. An imagefocus analyzer, preferably embodied as processor 132, is preferablyoperative to perform image focus analysis on the series of images, inorder to provide a focus score based on portions of each image and toascertain a focus distance based on the focus score. It is appreciatedthat processor 132 thus preferably operates as a focus distanceascertainer, ascertaining a focus distance based on a differentiatedimage, such as the image of FIG. 8B.

It is appreciated that the focus score may be calculated based only onthose portions of the depth differentiated image, such as the image ofFIG. 8B, at a depth at or above a predetermined depth threshold, in thecase of protruding copper traces 1110. Alternatively, the focus scoremay be calculated based only on those portions of the depthdifferentiated image at depths below a predetermined depth threshold,for example in the case of copper traces 1110 being embedded withinsubstrate 1109.

In this case, a focus score is preferably calculated on a pixel by pixelbasis in each of the images acquired under lighting conditions providedby feature focusing illuminator 150, the focus score being calculatedonly for those pixels identified in the segmented depth differentiatedimage, such as the image of FIG. 8B, as corresponding to regions ofinterest. It is appreciated that in order to generate a range image, thefocus score is preferably calculated for each pixel, in order toascertain the optimum focal height corresponding to maximum measuredfeature texture in that pixel. It is noted that in contrast to the focalscore calculation described hereinabove with reference to system 100, anoverall focus score, based on the sum of the focus scores of all thepixels in the region of interest in each image, is preferably notcalculated in this embodiment.

In the case of copper region 1110 on substrate 1109, by way of example,each pixel identified in the depth differentiated image, such as theimage of FIG. 8B, as corresponding to copper region 1110 is assigned afocus score based on an appropriate local texture measure such as thegradient magnitude or any other suitable focus measure known in the art.Pixels in the depth differentiated image, such as the image of FIG. 8B,identified as regions of non-interest, corresponding to substrate 1109in the illustrated embodiment, are assigned a focus score of zero. It isappreciated that the focus score is not calculated for those portions ofeach image below the predetermined brightness threshold, which portionsin this case correspond to substrate 1109.

The focus score obtained for each pixel may be plotted as a function ofthe focal height of lens 124, as illustrated in FIG. 9. As seen in FIG.9, a first trace 1202 represents variation of focal score with focalheight in the case of a pixel corresponding to a first indent 1204 seenin FIG. 8A, wherein a highest focal score of 100 is seen to correspondto an absolute focal height of approximately 6486 μm. As further seen inFIG. 9, a second trace 1206 represents variation of focal score withfocal height in the case of another pixel corresponding to a secondindent 1208. In this case, second indent 1208 is not as deep as firstindent 1204 represented by first trace 1202. As is appreciated from acomparison of first and second traces 1202 and 1206, the height at whichthe maximal focal score in the case of the second indent 1208 occurs isshifted with respect to that of the first indent 1204 due to thedifferences in depth therebetween.

Based on functions such as those illustrated in FIG. 9, a height imagemay be created wherein each pixel is assigned a value equal to the focalheight at which that pixel was found to have its highest focus score.Such a height image is shown in FIG. 8C where the gray color scalecorresponds to the pixel height in microns. As seen in FIG. 8C, graypixels in region 1110 represent higher regions and white pixels inregions 1204 and 1208 represent lower regions. Black pixels in region1109 correspond to pixels for which no focus score was calculated, sincethese pixels were identified as belonging to regions of non-interest,based on the segmented depth differentiated image, such as the image ofFIG. 8B.

It is appreciated that the height or range image of FIG. 8C may befurther analyzed in order to find the depth of indents 1204 and 1208relative to the bulk of copper region 1110.

It is understood that in the above-described approaches, the focalmetric based on which autofocusing is achieved is applied to thefeatures of interest only and is preferably confined within theboundaries of the features of interest. This is in contrast toconventional autofocusing methods wherein a focal metric is typicallyderived over the entire field of view of a camera and is thus heavilyinfluenced by the shape and size of various features, rather than bydepth alone, as is the case in the present invention.

Reference is now made to FIG. 10, which is a simplified schematicillustration of an optical processing system including depthdifferentiating functionality, constructed and operative in accordancewith a further preferred embodiment of the present invention.

As seen in FIG. 10, there is provided an optical imaging system 1300,preferably including an optical imaging head 1302 mounted on a chassis1304. Chassis 1304 preferably includes a table 1306 adapted forplacement thereon of an object 1308 to be imaged. Optical imaging system1300 is preferably operative to provide a depth profile image of object1308, for example for the purposes of inspection or processing of object1308.

Object 1308 is preferably a non-planar object comprising physicalfeatures at more than one physical depth. Here, by way of example,object 1308 is shown to be embodied as a PCB including a non-conductivesubstrate 1309 having metallic traces 1310 formed thereon, whichmetallic traces 1310 may be embedded or may protrude with respect to asurface of substrate 1309. It is appreciated, however, that opticalimaging head 1302 may be used to acquire images of any suitable targetor scene having physical features at more than one physical height ordepth including, but not limited to, PCBs, wafer dies, assembled PCBs,flat panel displays and solar energy wafers.

For inspection purposes, it is often desirable to generate atwo-dimensional image of object 1308, wherein the metallic traces 1310are clearly distinguished from substrate 1309 based on differences inoptical properties therebetween.

In some cases, it may also be desirable to generate a three-dimensionaldepth profile of a feature of interest included in object 1308, whichfeature of interest is at a different physical height or depth withrespect to other features of object 1308. For example, in the case ofsubstrate 1309, it may be desirable to generate a depth profile image ofmetallic traces 1310 for the purposes of inspection thereof.

It is a particular feature of a preferred embodiment of the presentinvention that optical imaging system 1300 includes a combined 2Dspatial and 3D range differentiator 1320 providing both spatiallysegmented and depth differentiated images of a feature of interest, suchas metallic traces 1310, notwithstanding the difference in physicaldepth between the feature of interest and other features, such assubstrate 1309. Particularly preferably, range differentiator 1320includes a 3D plenoptic camera 1321 for generating a depth profile imageof the feature of interest.

Range differentiator 1320 preferably includes an image generatoroperative to provide an image of a scene at various physical depths,here embodied, by way of example, as including an illumination module1322 for illuminating object 1308. Illumination provided by illuminationmodule 1322 is preferably directed towards object 1308 by way of a lensportion 1324. Light emanating from object 1308 is preferably directedtowards a two-dimensional imaging camera 1330, as well as towardsplenoptic camera 1321, via a beam splitter 1332.

Illuminator module 1322 preferably operates in two modes, a 2D mode anda 3D mode. In a 2D mode of operation, object 1308 is preferably imagedby two-dimensional imaging camera 1330 under illumination conditions inwhich the feature of interest is clearly distinguishable from the otherfeatures of object 1308 having a different physical depth range than thefeature of interest. Such illumination may be termed feature specificillumination and may be provided, by way of example only, by a brightfield illuminator 1340 and a dark field illuminator 1342 included inillumination module 1322. Bright field illuminator 1340 of illuminationmodule 1322 in combination with dark field illuminator 1342 ofillumination module 1322 may be considered to comprise a first portionof an image generator, delivering combined bright field and dark fieldillumination modalities.

Under a combination of bright and dark field illumination provided bybright field illuminator 1342 and dark field illuminator 1342,non-conductive substrate 1309 exhibits reduced reflectance in comparisonwith the reflectance exhibited by metallic traces 1310. An exemplaryimage of substrate 1309 and metallic traces 1310 thereon under featurespecific dark and bright field illumination conditions is shown in

FIG. 11A. As seen in FIG. 11A, non-conductive substrate 1309 has a darkappearance relative to the metallic traces 1310 due to the lowerreflectance thereof whereas metallic traces 1310 have a lighterappearance relative to substrate 1309. Non-conductive substrate 1309 isthus clearly distinguishable from metallic traces 1310 in the image ofFIG. 11A. Furthermore, as a result of the opacity of substrate 1309,additional layers of PCB 1308 that may lie beneath substrate 1309 areobscured and thereby do not appear in the image of FIG. 11A, thussimplifying subsequent image processing.

Following the generation of an initial feature specific image, such asthat shown in FIG. 11A, a depth differentiated or segmented image ispreferably generated, which segmented image is based on the initialfeature specific image. An exemplary segmented image based on thefeature specific image of FIG. 11A is illustrated in FIG. 11B. In thesegmented image of FIG. 11B, pixels corresponding to bright metallictraces 1310 are marked in white, distinguishing these pixels from pixelscorresponding to darker substrate regions 1309 which are marked inblack. Preferably, a predetermined threshold for level of pixelbrightness may be applied in order to distinguish between bright pixelscorresponding to metallic traces 1310 and darker pixels corresponding tobackground substrate 1309.

It is understood that the segmented image of FIG. 11B thus effectivelyforms a depth differentiated image, in which those portions of thefeature specific image of FIG. 11A at depths below a predeterminedthreshold, here, by way of example corresponding to substrate 1309, aredistinguished from those portions of the feature specific image of FIG.11A at depths above a predetermined threshold, here, by way of example,corresponding to metallic traces 1310. It is appreciated that thedifferentiation between portions of the feature specific image of FIG.11A at different physical depths is based on the difference in opticalproperties therebetween, and more specifically the difference inreflectance under dark and bright field illumination therebetween, andis independent of the physical shapes of the features.

The generation of the segmented mask image of FIG. 11B may beautomatically carried out by computing functionality included in aprocessor (not shown) forming part of system 1300. It is appreciatedthat the processor thus preferably operates as a depth differentiator,operative to distinguish portions of an initial feature specific imageobtained under illumination by a first imaging modality, such as theimage of FIG. 11A, at depths below a predetermined threshold,irrespective of a shape of the portions, and to provide a depthdifferentiated image, such as the depth differentiated image of FIG.11B.

It is appreciated that the feature of interest may be identifiable by auser in the feature specific images of FIGS. 11A and 11B as well aspreferably being a machine identifiable feature, the presence of whichmachine identifiable feature in the feature specific images may bedetected based on the appearance thereof, irrespective of the shape ofthe feature.

In the 3D mode of operation of system 1300, following the generation ofa segmented image such as that shown in FIG. 11B, object 1308 ispreferably imaged by plenoptic camera 1321 under illumination conditionsbest suited for enhancing the imaged texture of the feature of interest,here embodied as metallic traces 1310. Such illumination may be termedfeature focusing illumination and is preferably provided here by darkfield illuminator 1342. Dark field illuminator 1342 may be considered tocomprise a second portion of an image generator, delivering a dark fieldillumination modality to object 1308.

An exemplary image illustrating the appearance of metallic traces 1310under dark field illumination only, in which a heightened texture ofmetallic traces 1310 is visible, in shown in FIG. 11C.

It is appreciated that although dark field illuminator 1342 is describedherein as contributing both to the feature specific illumination andfeature focusing illumination, the feature specific illumination andfeature focusing illumination may alternatively be provided by disparateillumination elements not having overlapping functionality.

Furthermore, it is appreciated that the image generation functionalityof range differentiator 1320 is not limited to the particular camera andillumination components described herein and rather may comprise anysuitable components functional to generate an image of a scene atvarious physical depths, in which features having different physicaldepths are differentiable based on the optical properties thereof andirrespective of the shape thereof.

In an exemplary embodiment, plenoptic camera 1321 preferably provides adepth profile image of those portions identified as being suspecteddefects based on the 2D segmented image, such as the image of FIG. 11B.It is appreciated that the nature of certain suspected defectsidentifiable in a 2D segmented image of the type shown in FIG. 11B maybe better ascertained by way of a depth profile image, as the truenature and criticality of the suspected defect is often only revealedupon identifying its 3D profile. Efficient 2D segmentation typicallyrequires, in addition to generating brightness differences between thesubstrate 1309 and metallic traces 1310, suppressing the texture of themetal traces. This is achieved by a proper combination and carefulbalancing of both bright and darkfield illuminations. In contrast, 3Dprofiling by the plenoptic camera 1321 is strongly dependent on surfacetexture, for example in deriving stereo disparity between adjacent microimages. Using darkfield illumination alone maximizes the contrast of thesurface texture of both the metallic traces 1310 and the substrate 1309,leading to accurate depth rendering by plenoptic camera 1321.

An exemplary image illustrating a depth profile of metallic traces 1310as acquired by plenoptic camera 1321 under dark field illuminationprovided by dark field illuminator 1342 is shown in FIG. 11D. It isappreciated that although the field of view over which the depth profileof FIG. 11D is acquired is greater than that of the initial andsegmented images of FIGS. 11A, 11B and 11C, the depth profiling ofmetallic traces 1310 may alternatively be confined to a smaller portionof the metallic traces 1310, such as in the region of a suspecteddefect, in order to ascertain the nature of the defect and classify thedefect accordingly. In this case, the processor may operate as a focusdistance ascertainer, ascertaining the focus distance at each point fordepth profiling of a region in which a suspected defect lies, based onthe depth differentiated image, such as the image of FIG. 11B.

In another preferred mode of operation of the combined 2D spatial and 3Ddepth range differentiator 1320, plenoptic camera 1321 may be employedto automatically focus 2D camera 1330 prior to acquiring of the 2D imagethereby.

In this autofocusing mode, the inspected object 1308 is preferablyinitially brought to a coarse focus of plenoptic camera 1321 underfeature-focusing illumination conditions, such as dark fieldillumination conditions preferably provided by dark field illuminator1342. Such a preliminary coarse focus may be based on systemoptimization and engineering parameters and may involve pre-calibrationof system 1300, as is well known by those skilled in the art. FIG. 12Ashows an exemplary coarsely focused image of a substrate 1410, asacquired by plenoptic camera 1321. In the illustrated embodiment,substrate 1410 is a silicon wafer, containing an abrupt height step1420, with laser inscribed pits 1430 thereon. A correspondingout-of-focus 2D image as received by 2D camera 1330 is shown in FIG.12B.

The coarsely focused image acquired by plenoptic camera 1321 may then beprocessed by computing functionality included in the processor of system1300, in order to derive a depth profile of the instant field of view ofsubstrate 1410. An exemplary depth differentiated profile image based onthe coarsely focused image of FIG. 12A is shown in FIG. 12C. It isappreciated that, in contrast to the example illustrated in FIGS.11A-11D, in this mode of operation of range differentiator 1320, thebright field illumination modality provided by bright field illuminator1340 preferably constitutes a first imaging illumination modality, underwhich illumination a depth differentiable image is preferably acquired.

Based on the depth profile image of FIG. 12C, the feature depth at which2D camera 1330 should optimally be focused may be selected. By way ofexample, in the case of substrate 1410, the optimal focal depth of 2Dcamera 1330 may be that depth corresponding to the height of the upperside 1440 of the step in the silicon wafer in the image of FIG. 12C. Asis appreciated by one skilled in the art, the depth of focus ofplenoptic camera 1321 typically straddles the depth of field of 2Dcamera 1330 and may be in the range of 2-4 times greater, such that theaccuracy of the depth profile analysis based on the plenoptic image ofFIG. 12A is at least as good as the accuracy achievable based on thedepth of focus on lens 1324.

2D camera 1330 may then be automatically focused on the upper side 1440of the silicon step at the optimal focus depth identified based on thedepth profile image of FIG. 12C and a focused 2D image of substrate 1410correspondingly acquired under feature specific bright fieldillumination conditions. It is noted that in this case the focusspecific illumination is the same as the feature specific illumination.This is a consequence of the optical reflection properties of both thesilicon wafer and the laser formed pits on its surface. An exemplaryautomatically focused 2D image acquired under feature specific brightfield illumination conditions is shown in FIG. 12D.

It is appreciated that following the automatically focused 2D imaging,additional 3D plenoptic imaging of object 1308 may be performed ifnecessary, for example for the purpose of better classifying the natureof suspected defects present in the 2D autofocused image, as describedhereinabove with reference to FIGS. 11C and 11D.

Reference is now made to FIG. 13, which is a simplified illustration ofan optical processing system including auto-focusing functionality,constructed and operative in accordance with a further preferredembodiment of the present invention, and to FIGS. 14A-14C, which aresimplified examples of images produced by a system of the typeillustrated in FIG. 13.

As seen in FIG. 13, there is provided an optical imaging system 1500including a projector module 1502 operative to project a pattern onto anobject 1508. Imaging system 1500 further preferably includes a camerasensor module 1510 operative to acquire an image of object 1508 when apattern is projected thereon by projector module 1502. Preferably,projector module 1502 and camera module 1510 are angled with respect toa longitudinal axis 1512 defined with respect to object 1508. Projectormodule 1502 in combination with camera module 1510 may be considered toform an image generator, operative to generate an image of object 1508.

Object 1508 is preferably a non-planar object comprising physicalfeatures at more than one physical depth including, but not limited to,PCBs, wafer dies, assembled PCBs, flat panel displays and solar energywafers. Alternatively, object 1508 may be embodied as any object orscene containing features at a range of physical depths.

In some cases, it may be desirable to generate a focused image of afeature of interest included in object 1508, which feature of interestis at a different physical height or depth with respect to otherfeatures of object 1508. This may be automatically achieved in system1500 by way of projecting a regularly repeating pattern, such as asinusoidal or binary moire fringe pattern, onto a surface of object 1508and analyzing the shift in phase of the projected fringes, as isdetailed herein below.

The operation of system 1500 may be best understood with reference tothe images generated thereby, examples of which images are presented inFIGS. 14A-14C.

Turning now to FIG. 14A, an image of a fringe pattern 1600, preferablyprojected by projector module 1502 onto a surface of object 1508, isillustrated. As seen in FIG. 14A, fringe pattern 1600 undergoes variablephase shifts depending on the surface topology of the features on object1508 upon which the fringe pattern falls. Computing functionalityincluded in a processor 1516 forming part of system 1500 may beoperative to compute, preferably in real time, the phase shift in fringepattern 1600 in order to derive at least the height of the physicalfeature upon which the fringe pattern is projected. Processor 1516 maybe operative as a depth differentiator, for differentiating portions ofthe images acquired by camera module 1510 at various physical heights,irrespective of the shape thereof. Fringe phase shift analysis carriedout by computing functionality included in processor 1516 may include,by way of example, a windowed Fourier transform. Additionally, processor1516 may also control the generation of fringe patterns projected byprojector module 1502.

The height of the physical feature is preferably computed relative tothe height of a reference target incorporated in system 1500. The heightof the reference target may be calibrated with respect to an additionalimaging functionality (not shown) of system 1500 maintained in focusrelative to object 1508 or may be calibrated with respect to camerasensor 1510.

A two-dimensional height map and a three-dimensional height map ofobject 1508 based on the projected fringe map of FIG. 14A arerespectively illustrated in FIGS. 14B and 14C. As seen in FIGS. 14B and14C, the shifts in phase of the projected fringe pattern may be used asa basis for segmenting object 1508 according to the relative heights ofthe physical features responsible for producing the corresponding shiftsin the phase pattern. A feature of given height may thus be selected foroptimum focusing thereupon, whilst features at heights other than theselected height are effectively ignored in subsequent image focusing. Itis appreciated that the height maps of FIGS. 14B and 14C thus constitutesegmented or depth differentiated images, based on which a depth offeatures selected for optimum focus thereon may be ascertained. Based onheight selection alone, autofocusing of camera 1510 may thus beperformed on features at a given height level, irrespective of a shapeof those features. The optimum focus distance may be ascertained by wayof processor 1516 based on the depth differentiated images of FIGS. 14Band 14C.

It is appreciated that the optimum spatial frequency of the fringepattern projected by projector module 1502 is preferably set by takinginto account and balancing several opposing requirements. The spatialfrequency of the fringe pattern is preferably selected so as to be lowenough to allow projection and imaging thereof with good contrast. Inaddition, the spatial frequency of the fringe pattern is preferablyselected so as to be high enough to allow sufficiently high resolutionheight differentiation. Furthermore, the inter-fringe spacing within thefringe pattern is preferably selected so as to be large enough toencompass the full expected depth of object 1508 without phaseambiguity. Preferably, the fringe pattern has a sufficiently low spatialfrequency such that shifts in phase thereof may be uniquely correlatedto the physical depths giving rise to such shifts, without phaseambiguity.

At least these various factors are preferably balanced in order toderive the optimum spatial frequency of the fringe pattern for aparticular imaging application.

System 1500 may be particularly well-suited for use in a closed-looptracking autofocus mode, wherein object 1508 is preferably scannedcontinuously. In a continuous scanning mode, projector module 1502 ispreferably strobed so as to operate in a pulsed mode, preferably insynchronization with the operation of camera module 1510. Alternatively,projector module 1502 may operate continuously, preferably inconjunction with a globally shuttered camera module 1510.

In use of system 1500 for continuous closed loop autofocusing operation,various operational parameters of system 1500 are preferably optimized.The temporal rate at which the height of object 1508 is sampled, by wayof the projection of fringe pattern 1600 thereon and subsequent analysisof phase shifts thereof, is preferably selected so as to be sufficientlyhigh to be suited to the scanning speed of object 1508 and the rate ofheight variations thereof. The operational frame rate of camera module1510 is preferably set in accordance with the height sampling rate.

Additionally, the elapsed time between fringe image acquisition bycamera module 1510 and the obtaining of an analyzed height map, whichtime delay may be termed the system latency, is preferably optimized.The system latency may be primarily dependent on the computingperformance of a system controller of system 1500. The system latency ispreferably set so as to be sufficiently short in order to avoid anexcessive lag in the operation of the autofocusing functionalityfollowing the fringe image acquisition, which excessive lag wouldotherwise lead to focusing errors of the imaging functionality.

In certain embodiments of the present invention, the pixel resolution ofcamera module 1510 may be set so as to optimize the performance ofsystem 1500. The fewer the imaging pixels of camera 1510, the higher thecamera frame rate operation and the shorter the processing time.Additionally or alternatively, rather than computing the phase shiftover the entirety of the images acquired by camera module 1510, thephase shift may only be computed within sparsely selected regions insidethe image frames outputted by camera module 1510, whereby processingtime may be accelerated. The number, size, aspect ratio and spacing ofthose regions within which the phase shift is computed may be selectedby taking into account physical or other characteristics of object 1508.

It will be appreciated by persons skilled in the art that the presentinvention is not limited by what has been particularly claimedhereinbelow. Rather, the scope of the invention includes variouscombinations and subcombinations of the features described hereinaboveas well as modifications and variations thereof as would occur topersons skilled in the art upon reading the forgoing description withreference to the drawings and which are not in the prior art.

1. A range differentiator useful for auto-focusing, said rangedifferentiator comprising: an image generator providing an image of ascene at various physical depths; a depth differentiator distinguishingportions of said image at depths below a predetermined threshold,irrespective of a shape of said portions, and providing a depthdifferentiated image; and a focus distance ascertainer ascertaining afocus distance based on said depth differentiated image.
 2. A rangedifferentiator according to claim 1, wherein said image generatorcomprises a feature specific illuminator for illuminating said sceneduring acquisition of said image.
 3. A range differentiator according toclaim 2, wherein said depth differentiator is operative to distinguishbetween said portions of said image at depths below said predeterminedthreshold and portions of said image at depths at or above saidpredetermined threshold based on differences in optical propertiestherebetween, under illumination by said feature specific illuminator.4. A range differentiator according to claim 2, wherein said featurespecific illuminator comprises a UV illumination source and said depthdifferentiator is operative to distinguish between said portions of saidimage based on differences in fluorescence therebetween.
 5. A rangedifferentiator according to claim 2, wherein said feature specificilluminator comprises dark field and bright field illumination sourcesand said depth differentiator is operative to distinguish between saidportions of said image based on differences in reflectance therebetween.6. A range differentiator according to claim 3, wherein said focusdistance ascertainer is operative to ascertain said focal distance basedon one of: said portions of said image at depths below saidpredetermined threshold; and said portions of said image at a depth ator above said predetermined threshold.
 7. A range differentiatoraccording to claim 1, and also comprising an image focus analyzeroperative to provide a focus score based on portions of said image at adepth at or above said predetermined threshold, said focus distanceascertainer being operative to ascertain said focus distance based onsaid focus score.
 8. A range differentiator according to claim 7,wherein said image focus analyzer comprises an illuminator forilluminating said scene with illumination for enhancing an imagedtexture of said portions of said image at a depth at or above saidpredetermined threshold.
 9. A range differentiator according to claim 8,wherein said illuminator comprises a dark field illuminator.
 10. A rangedifferentiator according to claim 7, and wherein said focus score isassigned irrespective of a shape of said portions.
 11. A rangedifferentiator according to claim 7, and wherein said focus score isindividually assigned for each pixel corresponding to said portions ofsaid image at a depth at or above said predetermined threshold.
 12. Arange differentiator according to claim 1, and wherein said portions ofsaid image at a depth at or above said predetermined threshold aremachine identifiable.
 13. A range differentiator according to claim 5,wherein said image generator comprises a camera and said depthdifferentiated image comprises a two-dimensional image of said scene.14. A range differentiator according to claim 2, wherein said imagegenerator comprises a plenoptic camera and said depth differentiatedimage comprises a three-dimensional image of said scene.
 15. A rangedifferentiator according to claim 14, wherein said feature specificilluminator comprises a dark field illuminator.
 16. A rangedifferentiator according to claim 1, wherein said image generatorcomprises a projector projecting a repeating pattern onto said scene andsaid depth differentiator comprises a phase analyzer operative toanalyze shifts in phase of said repeating pattern and derive a map ofsaid physical depths based on said shifts in phase, said map formingsaid depth differentiated image.
 17. A range differentiator according toclaim 16, wherein said focus distance ascertainer is operative toascertain said focus distance based on at least one of said physicaldepths.
 18. A range differentiator according to claim 16, wherein saidrepeating pattern comprises at least one of a sinusoidal repeatingpattern and a binary repeating pattern.
 19. A range differentiatoraccording to claim 18, wherein said repeating pattern has a sufficientlylow spatial frequency such that said phase analyzer is operative touniquely correlate said shifts in phase to said physical depths.
 20. Arange differentiator according to claim 18, wherein said map of saidphysical depths is one of a two dimensional map and a three dimensionalmap.
 21. A range differentiator useful for auto-focusing, said rangedifferentiator comprising: an image generator providing an image of ascene at various physical depths; a depth differentiator distinguishingportions of said image at depths below a predetermined threshold; animage focus analyzer operative to provide a focus score based onportions of said image at a depth at or above said predeterminedthreshold; and a focus distance ascertainer ascertaining a focusdistance based on said focus score.
 22. A range differentiator accordingto claim 21, wherein said image generator comprises a feature specificilluminator for illuminating said scene during acquisition of saidimage.
 23. A range differentiator according to claim 22, wherein saidfeature specific illuminator comprises a UV illumination source and saiddepth differentiator distinguishes portions of said image based ondifferences in fluorescence therebetween.
 24. A range differentiatoraccording to claim 22, wherein said feature specific illuminatorcomprises a combined dark field and bright field illuminator and saiddepth differentiator distinguishes portions of said image based ondifferences in reflectance therebetween.
 25. A range differentiatoraccording to claim 21, wherein said image focus analyzer comprises anilluminator for illuminating said scene with illumination for enhancingan imaged texture of said portions of said image at a depth at or abovesaid predetermined threshold.
 26. A range differentiator according toclaim 25, wherein said illuminator comprises a dark field illuminator.27. A range differentiator according to claim 25, wherein saidilluminator and said feature specific illuminator share at least onecommon illumination component.
 28. A range differentiator according toclaim 21, and wherein said focus score is assigned irrespective of ashape of said portions.
 29. A range differentiator according to claim21, and wherein said focus score is individually assigned for each pixelcorresponding to said portions of said image at a depth at or above saidpredetermined threshold.
 30. A range differentiator according to claim21, and wherein said portions of said image at a depth at or above saidpredetermined threshold are machine identifiable.
 31. A rangedifferentiator useful for auto-focusing, said range differentiatorcomprising: a target identifier comprising a user interface enabling auser to identify a machine identifiable feature of an object in animage; a feature detector operative to identify at least one occurrenceof said machine identifiable feature in an image irrespective of a shapeof said feature; and a focus distance ascertainer ascertaining a focaldistance to said machine identifiable feature.
 32. A rangedifferentiator according to claim 31 and also comprising a featurespecific illuminator for illuminating said object during acquisition ofsaid image.
 33. A range differentiator according to claim 32, whereinsaid feature specific illuminator comprises a UV illumination source andsaid feature identifier identifies said machine identifiable featurebased on fluorescence thereof.
 34. A range differentiator according toclaim 32, wherein said feature specific illuminator comprises a combineddark field and bright field illuminator and said feature identifieridentifies said machine identifiable feature based on reflectancethereof.
 35. A range differentiator according to claim 32, wherein rangeascertainer comprises an illuminator for illuminating said object withillumination for enhancing an imaged texture of said feature of saidobject in said image.
 36. A range differentiator according to claim 35,wherein said illuminator comprises a dark field illuminator.
 37. A rangedifferentiator according to claim 35, wherein said illuminator and saidfeature specific illuminator share at least one common illuminationcomponent.
 38. A range differentiator according to claim 31, whereinsaid feature of said object comprises a conductive feature.
 39. A rangedifferentiator according to claim 38, wherein said feature of saidobject comprises an indent in said conductive feature.
 40. A rangedifferentiator useful for auto-focusing, said range differentiatorcomprising: a first image generator comprising a first imaging modalityand providing a first image of a scene at various physical depths; adepth differentiator distinguishing portions of said first image atdepths below a predetermined threshold and providing a depthdifferentiated image; a focus distance ascertainer ascertaining a focaldistance based on said depth differentiated image; and a second imagegenerator comprising a second imaging modality and providing a secondimage of said scene automatically focused at said focal distance.
 41. Arange differentiator according to claim 40, wherein said first imagingmodality comprises combined bright and dark field illumination and saidsecond imaging modality comprises dark field illumination.
 42. A rangedifferentiator according to claim 41, wherein said second imagegenerator comprises a plenoptic camera.
 43. A range differentiatoraccording to claim 40, wherein said first imaging modality comprisesdark field illumination and said second imaging modality comprisescombined bright and dark field illumination.
 44. A range differentiatoraccording to claim 43, wherein said first image generator comprises aplenoptic camera.
 45. A range differentiator useful for auto-focusing,said range differentiator comprising: a projector projecting a repeatingpattern onto an object comprising features of various physical depths; asensor acquiring an image of said object having said repeating patternprojected thereon; a phase analyzer analyzing shifts in phase of saidrepeating pattern and deriving a map of said physical depths of saidfeatures based on said shifts in phase; and a focus analyzerascertaining a focus distance to at least one of said features.
 46. Arange differentiator according to claim 45, wherein said repeatingpattern comprises at least one of a sinusoidal repeating pattern and abinary repeating pattern.
 47. A range differentiator according to claim45, wherein said repeating pattern has a sufficiently low spatialfrequency such that said phase analyzer is operative to uniquelycorrelate said shifts in phase to said physical depths.
 48. A rangedifferentiator according to claims 45, wherein said map of said physicaldepths is one of a two dimensional map or a three dimensional map.